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Out  work  addresses  the  combined  application  of  AI  techniques  and  true  3D  interaction  and 
display  devices  to  generate  knowledge-based  virtual  worlds.  The  goal  is  to  develop  virtual 
environments  that  present  high-quality  explanations  of  physical  tasks  such  as  those  involved 
in  equipment  maintenance  and  repair  or  training.  In  our  research,  we  are  exploring  issues  in 
the  automated  design  of  both  self-contained  virtual  worlds  and  “augmented  reality”  (in  which 
generated  graphics  augments  the  user’s  view  of  the  real  world). 
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During  the  first  quarter  of  our  three- year  contract,  which  started  in  September  1991,  we 
began  to  build  some  of  the  new  software  and  hardware  infrastructure  needed  to  support  our 
research.  In  part,  we  are  extending  our  IBIS  (Intent-Based  Illustration  System)  knowledge- 
based  graphics  generation  system  [Seligmann  &  Feiner  91].  The  current  IBIS  designs  static 
images  and  user-controlled  animation  that  describe  maintenance  and  repair  tasks.  We  are 
developing  a  new  version  that  can  work  with  a  prototype  see-through  head-mounted  display 
that  we  desined  during  the  summer  of  1991  [Feiner  &  Shamash  91].  The  prototype  display 
uses  a  Reflection  Technology  Private  Eye,  a  lightweight  display  whose  optics  produce  a 
virtual  image  that  can  be  focused  at  a  user-controlled  distance.  A  Logitech  3D  sensor 
monitors  head  position  and  orientation  and  a  mirror  beam  splitter  merges  the  display’s  image 
with  that  of  the  surrounding  world. 

Our  accomplishments  this  quarter  included: 

•  design  of  illustrations  as  overlays.  IBIS’s  design  rule  base  was  developed  to 
synthesize  the  entirety  of  an  illustration.  In  contrast,  the  work  performed  during 
the  first  quarter  of  our  contract  explores  the  automated  design  of  material  that 
supplements  the  user’s  view  of  the  surrounding  world.  We  began  the 
development  of  a  preliminary  set  of  design  rules  that  address  this  difference.  For 
example,  one  rule  states  that  if  IBIS  is  told  to  display  an  object  and  it  determines 
that  the  actual  object  is  already  within  the  user’s  view  of  the  physical  world  and 
not  blocked  by  other  objects,  nothing  needs  to  be  done  (since  the  user  can  see 
the  actual  object).  Otherwise,  if  the  object  is  within  the  user’s  view  and  blocked 
then  a  depiction  of  it  can  be  overlaid  on  the  user’s  view  to  allow  the  user  to  “see 
through”  the  blocking  objects.  Furthermore,  if  the  object  is  not  within  the  user’s 
field  of  view,  the  user  must  be  instructed  to  find  it:  one  approach  that  we  have 
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taken  is  the  use  of  a  textual  callout  whose  leader  line  points  in  the  direction  of 
the  object. 

•  3D  sensor  support  software.  We  developed  support  software  for  the  Logitech 
3D  sensor,  a  relatively  inexpensive  ultrasonic  3D  position  and  orientation  sensor 
announced  over  the  summer.  (An  earlier  version  of  our  head-mounted  display 
used  a  Polhemus  sensor.) 

•  Multi-ported  display-list  server.  We  designed  and  implemented  a  first  version 
of  a  multi-ported  3D  display  list  server.  The  server  allows  processes  associated 
with  multiple  3D  motion  sensors  to  monitor  the  motion  of  the  user’s  head  and  of 
other  objects  in  the  physical  world  and  modify  the  display  list  accordingly  This 
makes  it  possible  to  synchronize  the  overlaid  graphics  with  the  user’s  view  of 
the  surrounding  physical  world. 

•  Registration  of  generated  material  with  the  surrounding  environment.  We 
developed  a  first  version  of  a  set  of  calibration  procedures  for  registering  3D 
overlaid  graphics  with  the  physical  world. 


A  paper  describing  this  preliminary  testbed  was  submitted  for  publication  [Feiner, 

MacIntyre,  Seligmann  91].  Another  paper  [Chin  &  Feiner  92],  submitted  in  early  September 
to  the  1992  Symposium  on  Interactive  3D  Graphics,  was  accepted.  This  describes  our  work 
on  a  fast  algorithm  for  computing  object-precision  penumbrae  cast  by  convex  area  light 
sources.  (While  this  latter  work  may  sound  unrelated  to  the  research  goals  of  our  contract,  it 
makes  it  possible  to  determine  efficiently  which  objects  obscure  others,  a  vital  step  in  the 
automated  design  of  effective  3D  graphics.) 
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